{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Action1  男女声音识别     \n",
    "\n",
    "数据集：voice.csv     3168个录制的声音样本（来自男性和女性演讲者），采集的频率范围是0hz-280hz，已经对数据进行了预处理     一共有21个属性值，请判断该声音是男还是女？     使用Accuracy作为评价标准      \n",
    "\n",
    "1、完成代码（30points）     2、分享经验（30points）     3、得分Top3（10points）  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>meanfreq</th>\n",
       "      <th>sd</th>\n",
       "      <th>median</th>\n",
       "      <th>Q25</th>\n",
       "      <th>Q75</th>\n",
       "      <th>IQR</th>\n",
       "      <th>skew</th>\n",
       "      <th>kurt</th>\n",
       "      <th>sp.ent</th>\n",
       "      <th>sfm</th>\n",
       "      <th>...</th>\n",
       "      <th>centroid</th>\n",
       "      <th>meanfun</th>\n",
       "      <th>minfun</th>\n",
       "      <th>maxfun</th>\n",
       "      <th>meandom</th>\n",
       "      <th>mindom</th>\n",
       "      <th>maxdom</th>\n",
       "      <th>dfrange</th>\n",
       "      <th>modindx</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.059781</td>\n",
       "      <td>0.064241</td>\n",
       "      <td>0.032027</td>\n",
       "      <td>0.015071</td>\n",
       "      <td>0.090193</td>\n",
       "      <td>0.075122</td>\n",
       "      <td>12.863462</td>\n",
       "      <td>274.402906</td>\n",
       "      <td>0.893369</td>\n",
       "      <td>0.491918</td>\n",
       "      <td>...</td>\n",
       "      <td>0.059781</td>\n",
       "      <td>0.084279</td>\n",
       "      <td>0.015702</td>\n",
       "      <td>0.275862</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.066009</td>\n",
       "      <td>0.067310</td>\n",
       "      <td>0.040229</td>\n",
       "      <td>0.019414</td>\n",
       "      <td>0.092666</td>\n",
       "      <td>0.073252</td>\n",
       "      <td>22.423285</td>\n",
       "      <td>634.613855</td>\n",
       "      <td>0.892193</td>\n",
       "      <td>0.513724</td>\n",
       "      <td>...</td>\n",
       "      <td>0.066009</td>\n",
       "      <td>0.107937</td>\n",
       "      <td>0.015826</td>\n",
       "      <td>0.250000</td>\n",
       "      <td>0.009014</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.054688</td>\n",
       "      <td>0.046875</td>\n",
       "      <td>0.052632</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.077316</td>\n",
       "      <td>0.083829</td>\n",
       "      <td>0.036718</td>\n",
       "      <td>0.008701</td>\n",
       "      <td>0.131908</td>\n",
       "      <td>0.123207</td>\n",
       "      <td>30.757155</td>\n",
       "      <td>1024.927705</td>\n",
       "      <td>0.846389</td>\n",
       "      <td>0.478905</td>\n",
       "      <td>...</td>\n",
       "      <td>0.077316</td>\n",
       "      <td>0.098706</td>\n",
       "      <td>0.015656</td>\n",
       "      <td>0.271186</td>\n",
       "      <td>0.007990</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.015625</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.046512</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.151228</td>\n",
       "      <td>0.072111</td>\n",
       "      <td>0.158011</td>\n",
       "      <td>0.096582</td>\n",
       "      <td>0.207955</td>\n",
       "      <td>0.111374</td>\n",
       "      <td>1.232831</td>\n",
       "      <td>4.177296</td>\n",
       "      <td>0.963322</td>\n",
       "      <td>0.727232</td>\n",
       "      <td>...</td>\n",
       "      <td>0.151228</td>\n",
       "      <td>0.088965</td>\n",
       "      <td>0.017798</td>\n",
       "      <td>0.250000</td>\n",
       "      <td>0.201497</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.562500</td>\n",
       "      <td>0.554688</td>\n",
       "      <td>0.247119</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.135120</td>\n",
       "      <td>0.079146</td>\n",
       "      <td>0.124656</td>\n",
       "      <td>0.078720</td>\n",
       "      <td>0.206045</td>\n",
       "      <td>0.127325</td>\n",
       "      <td>1.101174</td>\n",
       "      <td>4.333713</td>\n",
       "      <td>0.971955</td>\n",
       "      <td>0.783568</td>\n",
       "      <td>...</td>\n",
       "      <td>0.135120</td>\n",
       "      <td>0.106398</td>\n",
       "      <td>0.016931</td>\n",
       "      <td>0.266667</td>\n",
       "      <td>0.712812</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>5.484375</td>\n",
       "      <td>5.476562</td>\n",
       "      <td>0.208274</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3163</th>\n",
       "      <td>0.131884</td>\n",
       "      <td>0.084734</td>\n",
       "      <td>0.153707</td>\n",
       "      <td>0.049285</td>\n",
       "      <td>0.201144</td>\n",
       "      <td>0.151859</td>\n",
       "      <td>1.762129</td>\n",
       "      <td>6.630383</td>\n",
       "      <td>0.962934</td>\n",
       "      <td>0.763182</td>\n",
       "      <td>...</td>\n",
       "      <td>0.131884</td>\n",
       "      <td>0.182790</td>\n",
       "      <td>0.083770</td>\n",
       "      <td>0.262295</td>\n",
       "      <td>0.832899</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>4.210938</td>\n",
       "      <td>4.203125</td>\n",
       "      <td>0.161929</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3164</th>\n",
       "      <td>0.116221</td>\n",
       "      <td>0.089221</td>\n",
       "      <td>0.076758</td>\n",
       "      <td>0.042718</td>\n",
       "      <td>0.204911</td>\n",
       "      <td>0.162193</td>\n",
       "      <td>0.693730</td>\n",
       "      <td>2.503954</td>\n",
       "      <td>0.960716</td>\n",
       "      <td>0.709570</td>\n",
       "      <td>...</td>\n",
       "      <td>0.116221</td>\n",
       "      <td>0.188980</td>\n",
       "      <td>0.034409</td>\n",
       "      <td>0.275862</td>\n",
       "      <td>0.909856</td>\n",
       "      <td>0.039062</td>\n",
       "      <td>3.679688</td>\n",
       "      <td>3.640625</td>\n",
       "      <td>0.277897</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3165</th>\n",
       "      <td>0.142056</td>\n",
       "      <td>0.095798</td>\n",
       "      <td>0.183731</td>\n",
       "      <td>0.033424</td>\n",
       "      <td>0.224360</td>\n",
       "      <td>0.190936</td>\n",
       "      <td>1.876502</td>\n",
       "      <td>6.604509</td>\n",
       "      <td>0.946854</td>\n",
       "      <td>0.654196</td>\n",
       "      <td>...</td>\n",
       "      <td>0.142056</td>\n",
       "      <td>0.209918</td>\n",
       "      <td>0.039506</td>\n",
       "      <td>0.275862</td>\n",
       "      <td>0.494271</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>2.937500</td>\n",
       "      <td>2.929688</td>\n",
       "      <td>0.194759</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3166</th>\n",
       "      <td>0.143659</td>\n",
       "      <td>0.090628</td>\n",
       "      <td>0.184976</td>\n",
       "      <td>0.043508</td>\n",
       "      <td>0.219943</td>\n",
       "      <td>0.176435</td>\n",
       "      <td>1.591065</td>\n",
       "      <td>5.388298</td>\n",
       "      <td>0.950436</td>\n",
       "      <td>0.675470</td>\n",
       "      <td>...</td>\n",
       "      <td>0.143659</td>\n",
       "      <td>0.172375</td>\n",
       "      <td>0.034483</td>\n",
       "      <td>0.250000</td>\n",
       "      <td>0.791360</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>3.593750</td>\n",
       "      <td>3.585938</td>\n",
       "      <td>0.311002</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3167</th>\n",
       "      <td>0.165509</td>\n",
       "      <td>0.092884</td>\n",
       "      <td>0.183044</td>\n",
       "      <td>0.070072</td>\n",
       "      <td>0.250827</td>\n",
       "      <td>0.180756</td>\n",
       "      <td>1.705029</td>\n",
       "      <td>5.769115</td>\n",
       "      <td>0.938829</td>\n",
       "      <td>0.601529</td>\n",
       "      <td>...</td>\n",
       "      <td>0.165509</td>\n",
       "      <td>0.185607</td>\n",
       "      <td>0.062257</td>\n",
       "      <td>0.271186</td>\n",
       "      <td>0.227022</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.554688</td>\n",
       "      <td>0.546875</td>\n",
       "      <td>0.350000</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3168 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      meanfreq        sd    median       Q25       Q75       IQR       skew  \\\n",
       "0     0.059781  0.064241  0.032027  0.015071  0.090193  0.075122  12.863462   \n",
       "1     0.066009  0.067310  0.040229  0.019414  0.092666  0.073252  22.423285   \n",
       "2     0.077316  0.083829  0.036718  0.008701  0.131908  0.123207  30.757155   \n",
       "3     0.151228  0.072111  0.158011  0.096582  0.207955  0.111374   1.232831   \n",
       "4     0.135120  0.079146  0.124656  0.078720  0.206045  0.127325   1.101174   \n",
       "...        ...       ...       ...       ...       ...       ...        ...   \n",
       "3163  0.131884  0.084734  0.153707  0.049285  0.201144  0.151859   1.762129   \n",
       "3164  0.116221  0.089221  0.076758  0.042718  0.204911  0.162193   0.693730   \n",
       "3165  0.142056  0.095798  0.183731  0.033424  0.224360  0.190936   1.876502   \n",
       "3166  0.143659  0.090628  0.184976  0.043508  0.219943  0.176435   1.591065   \n",
       "3167  0.165509  0.092884  0.183044  0.070072  0.250827  0.180756   1.705029   \n",
       "\n",
       "             kurt    sp.ent       sfm  ...  centroid   meanfun    minfun  \\\n",
       "0      274.402906  0.893369  0.491918  ...  0.059781  0.084279  0.015702   \n",
       "1      634.613855  0.892193  0.513724  ...  0.066009  0.107937  0.015826   \n",
       "2     1024.927705  0.846389  0.478905  ...  0.077316  0.098706  0.015656   \n",
       "3        4.177296  0.963322  0.727232  ...  0.151228  0.088965  0.017798   \n",
       "4        4.333713  0.971955  0.783568  ...  0.135120  0.106398  0.016931   \n",
       "...           ...       ...       ...  ...       ...       ...       ...   \n",
       "3163     6.630383  0.962934  0.763182  ...  0.131884  0.182790  0.083770   \n",
       "3164     2.503954  0.960716  0.709570  ...  0.116221  0.188980  0.034409   \n",
       "3165     6.604509  0.946854  0.654196  ...  0.142056  0.209918  0.039506   \n",
       "3166     5.388298  0.950436  0.675470  ...  0.143659  0.172375  0.034483   \n",
       "3167     5.769115  0.938829  0.601529  ...  0.165509  0.185607  0.062257   \n",
       "\n",
       "        maxfun   meandom    mindom    maxdom   dfrange   modindx   label  \n",
       "0     0.275862  0.007812  0.007812  0.007812  0.000000  0.000000    male  \n",
       "1     0.250000  0.009014  0.007812  0.054688  0.046875  0.052632    male  \n",
       "2     0.271186  0.007990  0.007812  0.015625  0.007812  0.046512    male  \n",
       "3     0.250000  0.201497  0.007812  0.562500  0.554688  0.247119    male  \n",
       "4     0.266667  0.712812  0.007812  5.484375  5.476562  0.208274    male  \n",
       "...        ...       ...       ...       ...       ...       ...     ...  \n",
       "3163  0.262295  0.832899  0.007812  4.210938  4.203125  0.161929  female  \n",
       "3164  0.275862  0.909856  0.039062  3.679688  3.640625  0.277897  female  \n",
       "3165  0.275862  0.494271  0.007812  2.937500  2.929688  0.194759  female  \n",
       "3166  0.250000  0.791360  0.007812  3.593750  3.585938  0.311002  female  \n",
       "3167  0.271186  0.227022  0.007812  0.554688  0.546875  0.350000  female  \n",
       "\n",
       "[3168 rows x 21 columns]"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "data = pd.read_csv('voice.csv')\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 3168 entries, 0 to 3167\n",
      "Data columns (total 21 columns):\n",
      " #   Column    Non-Null Count  Dtype  \n",
      "---  ------    --------------  -----  \n",
      " 0   meanfreq  3168 non-null   float64\n",
      " 1   sd        3168 non-null   float64\n",
      " 2   median    3168 non-null   float64\n",
      " 3   Q25       3168 non-null   float64\n",
      " 4   Q75       3168 non-null   float64\n",
      " 5   IQR       3168 non-null   float64\n",
      " 6   skew      3168 non-null   float64\n",
      " 7   kurt      3168 non-null   float64\n",
      " 8   sp.ent    3168 non-null   float64\n",
      " 9   sfm       3168 non-null   float64\n",
      " 10  mode      3168 non-null   float64\n",
      " 11  centroid  3168 non-null   float64\n",
      " 12  meanfun   3168 non-null   float64\n",
      " 13  minfun    3168 non-null   float64\n",
      " 14  maxfun    3168 non-null   float64\n",
      " 15  meandom   3168 non-null   float64\n",
      " 16  mindom    3168 non-null   float64\n",
      " 17  maxdom    3168 non-null   float64\n",
      " 18  dfrange   3168 non-null   float64\n",
      " 19  modindx   3168 non-null   float64\n",
      " 20  label     3168 non-null   object \n",
      "dtypes: float64(20), object(1)\n",
      "memory usage: 519.9+ KB\n"
     ]
    }
   ],
   "source": [
    "data.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "female    1584\n",
       "male      1584\n",
       "Name: label, dtype: int64"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.label.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['label'] = data['label'].map(lambda x:1 if x=='male' else 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "meanfreq    0\n",
       "sd          0\n",
       "median      0\n",
       "Q25         0\n",
       "Q75         0\n",
       "IQR         0\n",
       "skew        0\n",
       "kurt        0\n",
       "sp.ent      0\n",
       "sfm         0\n",
       "mode        0\n",
       "centroid    0\n",
       "meanfun     0\n",
       "minfun      0\n",
       "maxfun      0\n",
       "meandom     0\n",
       "mindom      0\n",
       "maxdom      0\n",
       "dfrange     0\n",
       "modindx     0\n",
       "label       0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 查看有没有nan值\n",
    "data.isna().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_X = data.drop(['label'],axis=1)\n",
    "data_Y = data['label']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "试过规范化，但是对结果没有任何影响，证明树模型不需要规范化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from sklearn.preprocessing import StandardScaler\n",
    "# scaler = StandardScaler()\n",
    "# data_X = scaler.fit_transform(data_X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 数据集切分\n",
    "from sklearn.model_selection import train_test_split\n",
    "x_train,x_test,y_train,y_test = train_test_split(data_X,data_Y,\n",
    "                                                 test_size = 0.27,\n",
    "                                                 random_state = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## XGBoost "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [],
   "source": [
    "import xgboost as xgb\n",
    "param = {'boosting_type':'gbdt',\n",
    "                         'objective' : 'binary:logistic', #任务目标\n",
    "                         'eval_metric' : 'auc', #评估指标\n",
    "                         'eta' : 0.01, #学习率\n",
    "                         'max_depth' : 15, #树最大深度\n",
    "                         'colsample_bytree':0.8, #设置在每次迭代中使用特征的比例\n",
    "                         'subsample': 0.9, #样本采样比例\n",
    "                         'subsample_freq': 8, #bagging的次数\n",
    "                         'alpha': 0.6, #L1正则\n",
    "                         'lambda': 0, #L2正则\n",
    "        }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[08:51:45] WARNING: C:\\Users\\Administrator\\workspace\\xgboost-win64_release_1.1.0\\src\\learner.cc:480: \n",
      "Parameters: { boosting_type, subsample_freq } might not be used.\n",
      "\n",
      "  This may not be accurate due to some parameters are only used in language bindings but\n",
      "  passed down to XGBoost core.  Or some parameters are not used but slip through this\n",
      "  verification. Please open an issue if you find above cases.\n",
      "\n",
      "\n",
      "[0]\ttrain-auc:0.99169\tvalid-auc:0.98634\n",
      "Multiple eval metrics have been passed: 'valid-auc' will be used for early stopping.\n",
      "\n",
      "Will train until valid-auc hasn't improved in 25 rounds.\n",
      "[25]\ttrain-auc:0.99953\tvalid-auc:0.99645\n",
      "[50]\ttrain-auc:0.99957\tvalid-auc:0.99709\n",
      "Stopping. Best iteration:\n",
      "[40]\ttrain-auc:0.99956\tvalid-auc:0.99732\n",
      "\n",
      "XGBoost预测结果： [0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1]\n"
     ]
    }
   ],
   "source": [
    "train_data = xgb.DMatrix(x_train,y_train)\n",
    "test_data = xgb.DMatrix(x_test,y_test)\n",
    "model = xgb.train(param,train_data,evals=[(train_data,'train'),(test_data,'valid')],\n",
    "                  num_boost_round=3000,early_stopping_rounds=25,verbose_eval=25)\n",
    "y_pred = model.predict(test_data)\n",
    "y_pred = [1 if i >=0.5 else 0 for i in y_pred]\n",
    "print('XGBoost预测结果：',y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBoost预测准确率： 0.977803738317757\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "print('XGBoost预测准确率：',accuracy_score(y_test,y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LightGBM "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [],
   "source": [
    "import lightgbm as lgb\n",
    "model = lgb.LGBMClassifier(\n",
    "        boosting_type=\"gbdt\", num_leaves=30, reg_alpha=0, reg_lambda=0.,\n",
    "    max_depth=-1, n_estimators=1500, objective='binary',metric= 'auc',\n",
    "    subsample=0.95, colsample_bytree=0.7, subsample_freq=1,\n",
    "    learning_rate=0.02, random_state=2017\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1]\ttraining's auc: 0.995877\tvalid_1's auc: 0.987692\n",
      "Training until validation scores don't improve for 200 rounds\n",
      "[2]\ttraining's auc: 0.997104\tvalid_1's auc: 0.989684\n",
      "[3]\ttraining's auc: 0.997866\tvalid_1's auc: 0.992172\n",
      "[4]\ttraining's auc: 0.998039\tvalid_1's auc: 0.992374\n",
      "[5]\ttraining's auc: 0.998649\tvalid_1's auc: 0.994878\n",
      "[6]\ttraining's auc: 0.998812\tvalid_1's auc: 0.996302\n",
      "[7]\ttraining's auc: 0.998882\tvalid_1's auc: 0.996846\n",
      "[8]\ttraining's auc: 0.998864\tvalid_1's auc: 0.996807\n",
      "[9]\ttraining's auc: 0.99893\tvalid_1's auc: 0.997004\n",
      "[10]\ttraining's auc: 0.998926\tvalid_1's auc: 0.996917\n",
      "[11]\ttraining's auc: 0.998947\tvalid_1's auc: 0.996922\n",
      "[12]\ttraining's auc: 0.998977\tvalid_1's auc: 0.996993\n",
      "[13]\ttraining's auc: 0.999051\tvalid_1's auc: 0.996917\n",
      "[14]\ttraining's auc: 0.999072\tvalid_1's auc: 0.996922\n",
      "[15]\ttraining's auc: 0.999105\tvalid_1's auc: 0.996977\n",
      "[16]\ttraining's auc: 0.999088\tvalid_1's auc: 0.996993\n",
      "[17]\ttraining's auc: 0.999075\tvalid_1's auc: 0.997092\n",
      "[18]\ttraining's auc: 0.99906\tvalid_1's auc: 0.996961\n",
      "[19]\ttraining's auc: 0.999075\tvalid_1's auc: 0.996982\n",
      "[20]\ttraining's auc: 0.999095\tvalid_1's auc: 0.996922\n",
      "[21]\ttraining's auc: 0.999088\tvalid_1's auc: 0.996884\n",
      "[22]\ttraining's auc: 0.99912\tvalid_1's auc: 0.996884\n",
      "[23]\ttraining's auc: 0.999125\tvalid_1's auc: 0.996846\n",
      "[24]\ttraining's auc: 0.999125\tvalid_1's auc: 0.996807\n",
      "[25]\ttraining's auc: 0.999151\tvalid_1's auc: 0.996906\n",
      "[26]\ttraining's auc: 0.999121\tvalid_1's auc: 0.996944\n",
      "[27]\ttraining's auc: 0.999137\tvalid_1's auc: 0.996944\n",
      "[28]\ttraining's auc: 0.999104\tvalid_1's auc: 0.996797\n",
      "[29]\ttraining's auc: 0.999101\tvalid_1's auc: 0.996687\n",
      "[30]\ttraining's auc: 0.999115\tvalid_1's auc: 0.996742\n",
      "[31]\ttraining's auc: 0.999143\tvalid_1's auc: 0.99684\n",
      "[32]\ttraining's auc: 0.999151\tvalid_1's auc: 0.996846\n",
      "[33]\ttraining's auc: 0.999176\tvalid_1's auc: 0.996939\n",
      "[34]\ttraining's auc: 0.999168\tvalid_1's auc: 0.996944\n",
      "[35]\ttraining's auc: 0.999227\tvalid_1's auc: 0.997026\n",
      "[36]\ttraining's auc: 0.999255\tvalid_1's auc: 0.996993\n",
      "[37]\ttraining's auc: 0.99926\tvalid_1's auc: 0.997064\n",
      "[38]\ttraining's auc: 0.999253\tvalid_1's auc: 0.997053\n",
      "[39]\ttraining's auc: 0.999277\tvalid_1's auc: 0.99707\n",
      "[40]\ttraining's auc: 0.999284\tvalid_1's auc: 0.997059\n",
      "[41]\ttraining's auc: 0.999291\tvalid_1's auc: 0.997103\n",
      "[42]\ttraining's auc: 0.999302\tvalid_1's auc: 0.997163\n",
      "[43]\ttraining's auc: 0.999313\tvalid_1's auc: 0.997146\n",
      "[44]\ttraining's auc: 0.999326\tvalid_1's auc: 0.997146\n",
      "[45]\ttraining's auc: 0.999347\tvalid_1's auc: 0.997141\n",
      "[46]\ttraining's auc: 0.999345\tvalid_1's auc: 0.997168\n",
      "[47]\ttraining's auc: 0.999358\tvalid_1's auc: 0.997261\n",
      "[48]\ttraining's auc: 0.999358\tvalid_1's auc: 0.997278\n",
      "[49]\ttraining's auc: 0.999354\tvalid_1's auc: 0.997272\n",
      "[50]\ttraining's auc: 0.999359\tvalid_1's auc: 0.997278\n",
      "[51]\ttraining's auc: 0.999363\tvalid_1's auc: 0.99731\n",
      "[52]\ttraining's auc: 0.999371\tvalid_1's auc: 0.997321\n",
      "[53]\ttraining's auc: 0.999386\tvalid_1's auc: 0.997338\n",
      "[54]\ttraining's auc: 0.999383\tvalid_1's auc: 0.997321\n",
      "[55]\ttraining's auc: 0.999395\tvalid_1's auc: 0.997283\n",
      "[56]\ttraining's auc: 0.999398\tvalid_1's auc: 0.997283\n",
      "[57]\ttraining's auc: 0.999404\tvalid_1's auc: 0.997294\n",
      "[58]\ttraining's auc: 0.999412\tvalid_1's auc: 0.997278\n",
      "[59]\ttraining's auc: 0.999426\tvalid_1's auc: 0.997278\n",
      "[60]\ttraining's auc: 0.99944\tvalid_1's auc: 0.997316\n",
      "[61]\ttraining's auc: 0.999436\tvalid_1's auc: 0.997305\n",
      "[62]\ttraining's auc: 0.999447\tvalid_1's auc: 0.997267\n",
      "[63]\ttraining's auc: 0.999454\tvalid_1's auc: 0.997294\n",
      "[64]\ttraining's auc: 0.999476\tvalid_1's auc: 0.997289\n",
      "[65]\ttraining's auc: 0.999482\tvalid_1's auc: 0.997278\n",
      "[66]\ttraining's auc: 0.999487\tvalid_1's auc: 0.997278\n",
      "[67]\ttraining's auc: 0.999492\tvalid_1's auc: 0.997272\n",
      "[68]\ttraining's auc: 0.999502\tvalid_1's auc: 0.997299\n",
      "[69]\ttraining's auc: 0.999511\tvalid_1's auc: 0.997332\n",
      "[70]\ttraining's auc: 0.999532\tvalid_1's auc: 0.997354\n",
      "[71]\ttraining's auc: 0.999548\tvalid_1's auc: 0.997458\n",
      "[72]\ttraining's auc: 0.999555\tvalid_1's auc: 0.997409\n",
      "[73]\ttraining's auc: 0.999553\tvalid_1's auc: 0.997403\n",
      "[74]\ttraining's auc: 0.99956\tvalid_1's auc: 0.997431\n",
      "[75]\ttraining's auc: 0.999576\tvalid_1's auc: 0.997469\n",
      "[76]\ttraining's auc: 0.999584\tvalid_1's auc: 0.99748\n",
      "[77]\ttraining's auc: 0.999593\tvalid_1's auc: 0.997458\n",
      "[78]\ttraining's auc: 0.999598\tvalid_1's auc: 0.997447\n",
      "[79]\ttraining's auc: 0.999607\tvalid_1's auc: 0.997474\n",
      "[80]\ttraining's auc: 0.999618\tvalid_1's auc: 0.997469\n",
      "[81]\ttraining's auc: 0.999621\tvalid_1's auc: 0.997458\n",
      "[82]\ttraining's auc: 0.999634\tvalid_1's auc: 0.997474\n",
      "[83]\ttraining's auc: 0.999633\tvalid_1's auc: 0.997496\n",
      "[84]\ttraining's auc: 0.99964\tvalid_1's auc: 0.997507\n",
      "[85]\ttraining's auc: 0.999643\tvalid_1's auc: 0.997513\n",
      "[86]\ttraining's auc: 0.999647\tvalid_1's auc: 0.997507\n",
      "[87]\ttraining's auc: 0.999655\tvalid_1's auc: 0.997551\n",
      "[88]\ttraining's auc: 0.999665\tvalid_1's auc: 0.997556\n",
      "[89]\ttraining's auc: 0.999668\tvalid_1's auc: 0.997551\n",
      "[90]\ttraining's auc: 0.999681\tvalid_1's auc: 0.997606\n",
      "[91]\ttraining's auc: 0.999689\tvalid_1's auc: 0.997627\n",
      "[92]\ttraining's auc: 0.999696\tvalid_1's auc: 0.997611\n",
      "[93]\ttraining's auc: 0.999704\tvalid_1's auc: 0.997611\n",
      "[94]\ttraining's auc: 0.99971\tvalid_1's auc: 0.997633\n",
      "[95]\ttraining's auc: 0.999716\tvalid_1's auc: 0.997627\n",
      "[96]\ttraining's auc: 0.999718\tvalid_1's auc: 0.997644\n",
      "[97]\ttraining's auc: 0.99972\tvalid_1's auc: 0.997627\n",
      "[98]\ttraining's auc: 0.999725\tvalid_1's auc: 0.9976\n",
      "[99]\ttraining's auc: 0.999728\tvalid_1's auc: 0.997622\n",
      "[100]\ttraining's auc: 0.999729\tvalid_1's auc: 0.997589\n",
      "[101]\ttraining's auc: 0.999739\tvalid_1's auc: 0.9976\n",
      "[102]\ttraining's auc: 0.999746\tvalid_1's auc: 0.997589\n",
      "[103]\ttraining's auc: 0.999751\tvalid_1's auc: 0.997611\n",
      "[104]\ttraining's auc: 0.999757\tvalid_1's auc: 0.997595\n",
      "[105]\ttraining's auc: 0.99976\tvalid_1's auc: 0.997584\n",
      "[106]\ttraining's auc: 0.999762\tvalid_1's auc: 0.997578\n",
      "[107]\ttraining's auc: 0.999773\tvalid_1's auc: 0.9976\n",
      "[108]\ttraining's auc: 0.999773\tvalid_1's auc: 0.997595\n",
      "[109]\ttraining's auc: 0.999778\tvalid_1's auc: 0.997589\n",
      "[110]\ttraining's auc: 0.999789\tvalid_1's auc: 0.997638\n",
      "[111]\ttraining's auc: 0.99979\tvalid_1's auc: 0.997627\n",
      "[112]\ttraining's auc: 0.999792\tvalid_1's auc: 0.997617\n",
      "[113]\ttraining's auc: 0.999794\tvalid_1's auc: 0.997627\n",
      "[114]\ttraining's auc: 0.9998\tvalid_1's auc: 0.997655\n",
      "[115]\ttraining's auc: 0.999807\tvalid_1's auc: 0.997699\n",
      "[116]\ttraining's auc: 0.999811\tvalid_1's auc: 0.997699\n",
      "[117]\ttraining's auc: 0.999813\tvalid_1's auc: 0.997699\n",
      "[118]\ttraining's auc: 0.999816\tvalid_1's auc: 0.997699\n",
      "[119]\ttraining's auc: 0.999826\tvalid_1's auc: 0.997715\n",
      "[120]\ttraining's auc: 0.999828\tvalid_1's auc: 0.997704\n",
      "[121]\ttraining's auc: 0.99983\tvalid_1's auc: 0.997699\n",
      "[122]\ttraining's auc: 0.999833\tvalid_1's auc: 0.997693\n",
      "[123]\ttraining's auc: 0.999839\tvalid_1's auc: 0.997699\n",
      "[124]\ttraining's auc: 0.999841\tvalid_1's auc: 0.997699\n",
      "[125]\ttraining's auc: 0.999844\tvalid_1's auc: 0.997693\n",
      "[126]\ttraining's auc: 0.99985\tvalid_1's auc: 0.99772\n",
      "[127]\ttraining's auc: 0.999855\tvalid_1's auc: 0.997726\n",
      "[128]\ttraining's auc: 0.999856\tvalid_1's auc: 0.997726\n",
      "[129]\ttraining's auc: 0.999855\tvalid_1's auc: 0.997731\n",
      "[130]\ttraining's auc: 0.999862\tvalid_1's auc: 0.997742\n",
      "[131]\ttraining's auc: 0.999865\tvalid_1's auc: 0.997748\n",
      "[132]\ttraining's auc: 0.999871\tvalid_1's auc: 0.997764\n",
      "[133]\ttraining's auc: 0.999876\tvalid_1's auc: 0.997759\n",
      "[134]\ttraining's auc: 0.999877\tvalid_1's auc: 0.99777\n",
      "[135]\ttraining's auc: 0.999882\tvalid_1's auc: 0.997808\n",
      "[136]\ttraining's auc: 0.999882\tvalid_1's auc: 0.997802\n",
      "[137]\ttraining's auc: 0.999887\tvalid_1's auc: 0.997786\n",
      "[138]\ttraining's auc: 0.999889\tvalid_1's auc: 0.997775\n",
      "[139]\ttraining's auc: 0.999891\tvalid_1's auc: 0.997775\n",
      "[140]\ttraining's auc: 0.999894\tvalid_1's auc: 0.997786\n",
      "[141]\ttraining's auc: 0.999894\tvalid_1's auc: 0.997819\n",
      "[142]\ttraining's auc: 0.9999\tvalid_1's auc: 0.997824\n",
      "[143]\ttraining's auc: 0.999902\tvalid_1's auc: 0.997868\n",
      "[144]\ttraining's auc: 0.999907\tvalid_1's auc: 0.997873\n",
      "[145]\ttraining's auc: 0.999911\tvalid_1's auc: 0.997863\n",
      "[146]\ttraining's auc: 0.999912\tvalid_1's auc: 0.997879\n",
      "[147]\ttraining's auc: 0.999918\tvalid_1's auc: 0.997884\n",
      "[148]\ttraining's auc: 0.999922\tvalid_1's auc: 0.997879\n",
      "[149]\ttraining's auc: 0.999922\tvalid_1's auc: 0.997895\n",
      "[150]\ttraining's auc: 0.999924\tvalid_1's auc: 0.997906\n",
      "[151]\ttraining's auc: 0.999927\tvalid_1's auc: 0.997923\n",
      "[152]\ttraining's auc: 0.999929\tvalid_1's auc: 0.997912\n",
      "[153]\ttraining's auc: 0.999932\tvalid_1's auc: 0.997923\n",
      "[154]\ttraining's auc: 0.999934\tvalid_1's auc: 0.99795\n",
      "[155]\ttraining's auc: 0.999939\tvalid_1's auc: 0.997955\n",
      "[156]\ttraining's auc: 0.999942\tvalid_1's auc: 0.997966\n",
      "[157]\ttraining's auc: 0.999943\tvalid_1's auc: 0.997955\n",
      "[158]\ttraining's auc: 0.999948\tvalid_1's auc: 0.997955\n",
      "[159]\ttraining's auc: 0.999949\tvalid_1's auc: 0.997972\n",
      "[160]\ttraining's auc: 0.99995\tvalid_1's auc: 0.997955\n",
      "[161]\ttraining's auc: 0.999951\tvalid_1's auc: 0.997945\n",
      "[162]\ttraining's auc: 0.999957\tvalid_1's auc: 0.997945\n",
      "[163]\ttraining's auc: 0.999958\tvalid_1's auc: 0.99795\n",
      "[164]\ttraining's auc: 0.99996\tvalid_1's auc: 0.99795\n",
      "[165]\ttraining's auc: 0.999963\tvalid_1's auc: 0.997961\n",
      "[166]\ttraining's auc: 0.999964\tvalid_1's auc: 0.997955\n",
      "[167]\ttraining's auc: 0.999966\tvalid_1's auc: 0.997966\n",
      "[168]\ttraining's auc: 0.999966\tvalid_1's auc: 0.997972\n",
      "[169]\ttraining's auc: 0.999967\tvalid_1's auc: 0.997961\n",
      "[170]\ttraining's auc: 0.999969\tvalid_1's auc: 0.997955\n",
      "[171]\ttraining's auc: 0.99997\tvalid_1's auc: 0.997972\n",
      "[172]\ttraining's auc: 0.999972\tvalid_1's auc: 0.99795\n",
      "[173]\ttraining's auc: 0.999972\tvalid_1's auc: 0.99795\n",
      "[174]\ttraining's auc: 0.999974\tvalid_1's auc: 0.997972\n",
      "[175]\ttraining's auc: 0.999976\tvalid_1's auc: 0.997966\n",
      "[176]\ttraining's auc: 0.999979\tvalid_1's auc: 0.997961\n",
      "[177]\ttraining's auc: 0.99998\tvalid_1's auc: 0.997961\n",
      "[178]\ttraining's auc: 0.999981\tvalid_1's auc: 0.997945\n",
      "[179]\ttraining's auc: 0.999981\tvalid_1's auc: 0.997945\n",
      "[180]\ttraining's auc: 0.999981\tvalid_1's auc: 0.997928\n",
      "[181]\ttraining's auc: 0.999981\tvalid_1's auc: 0.997934\n",
      "[182]\ttraining's auc: 0.999981\tvalid_1's auc: 0.997928\n",
      "[183]\ttraining's auc: 0.999983\tvalid_1's auc: 0.997961\n",
      "[184]\ttraining's auc: 0.999983\tvalid_1's auc: 0.997983\n",
      "[185]\ttraining's auc: 0.999984\tvalid_1's auc: 0.997972\n",
      "[186]\ttraining's auc: 0.999985\tvalid_1's auc: 0.997988\n",
      "[187]\ttraining's auc: 0.999985\tvalid_1's auc: 0.997972\n",
      "[188]\ttraining's auc: 0.999986\tvalid_1's auc: 0.997983\n",
      "[189]\ttraining's auc: 0.999987\tvalid_1's auc: 0.997972\n",
      "[190]\ttraining's auc: 0.999987\tvalid_1's auc: 0.997977\n",
      "[191]\ttraining's auc: 0.999987\tvalid_1's auc: 0.997988\n",
      "[192]\ttraining's auc: 0.999988\tvalid_1's auc: 0.997994\n",
      "[193]\ttraining's auc: 0.99999\tvalid_1's auc: 0.997988\n",
      "[194]\ttraining's auc: 0.99999\tvalid_1's auc: 0.997977\n",
      "[195]\ttraining's auc: 0.99999\tvalid_1's auc: 0.997988\n",
      "[196]\ttraining's auc: 0.999992\tvalid_1's auc: 0.997994\n",
      "[197]\ttraining's auc: 0.999992\tvalid_1's auc: 0.997994\n",
      "[198]\ttraining's auc: 0.999993\tvalid_1's auc: 0.997999\n",
      "[199]\ttraining's auc: 0.999993\tvalid_1's auc: 0.997994\n",
      "[200]\ttraining's auc: 0.999993\tvalid_1's auc: 0.997994\n",
      "[201]\ttraining's auc: 0.999994\tvalid_1's auc: 0.997988\n",
      "[202]\ttraining's auc: 0.999994\tvalid_1's auc: 0.998005\n",
      "[203]\ttraining's auc: 0.999994\tvalid_1's auc: 0.99801\n",
      "[204]\ttraining's auc: 0.999996\tvalid_1's auc: 0.998005\n",
      "[205]\ttraining's auc: 0.999996\tvalid_1's auc: 0.99801\n",
      "[206]\ttraining's auc: 0.999997\tvalid_1's auc: 0.998005\n",
      "[207]\ttraining's auc: 0.999998\tvalid_1's auc: 0.998005\n",
      "[208]\ttraining's auc: 0.999999\tvalid_1's auc: 0.998016\n",
      "[209]\ttraining's auc: 0.999999\tvalid_1's auc: 0.997999\n",
      "[210]\ttraining's auc: 0.999999\tvalid_1's auc: 0.99801\n",
      "[211]\ttraining's auc: 0.999999\tvalid_1's auc: 0.998021\n",
      "[212]\ttraining's auc: 0.999999\tvalid_1's auc: 0.998016\n",
      "[213]\ttraining's auc: 0.999999\tvalid_1's auc: 0.998021\n",
      "[214]\ttraining's auc: 0.999999\tvalid_1's auc: 0.998021\n",
      "[215]\ttraining's auc: 0.999999\tvalid_1's auc: 0.998016\n",
      "[216]\ttraining's auc: 0.999999\tvalid_1's auc: 0.998016\n",
      "[217]\ttraining's auc: 0.999999\tvalid_1's auc: 0.998016\n",
      "[218]\ttraining's auc: 0.999999\tvalid_1's auc: 0.99801\n",
      "[219]\ttraining's auc: 0.999999\tvalid_1's auc: 0.998005\n",
      "[220]\ttraining's auc: 0.999999\tvalid_1's auc: 0.997999\n",
      "[221]\ttraining's auc: 0.999999\tvalid_1's auc: 0.997994\n",
      "[222]\ttraining's auc: 0.999999\tvalid_1's auc: 0.997999\n",
      "[223]\ttraining's auc: 0.999999\tvalid_1's auc: 0.997988\n",
      "[224]\ttraining's auc: 0.999999\tvalid_1's auc: 0.997977\n",
      "[225]\ttraining's auc: 0.999999\tvalid_1's auc: 0.997994\n",
      "[226]\ttraining's auc: 0.999999\tvalid_1's auc: 0.997999\n",
      "[227]\ttraining's auc: 0.999999\tvalid_1's auc: 0.998021\n",
      "[228]\ttraining's auc: 0.999999\tvalid_1's auc: 0.998054\n",
      "[229]\ttraining's auc: 1\tvalid_1's auc: 0.998037\n",
      "[230]\ttraining's auc: 1\tvalid_1's auc: 0.998027\n",
      "[231]\ttraining's auc: 1\tvalid_1's auc: 0.998032\n",
      "[232]\ttraining's auc: 1\tvalid_1's auc: 0.998043\n",
      "[233]\ttraining's auc: 1\tvalid_1's auc: 0.998037\n",
      "[234]\ttraining's auc: 1\tvalid_1's auc: 0.998043\n",
      "[235]\ttraining's auc: 1\tvalid_1's auc: 0.998048\n",
      "[236]\ttraining's auc: 1\tvalid_1's auc: 0.998065\n",
      "[237]\ttraining's auc: 1\tvalid_1's auc: 0.998076\n",
      "[238]\ttraining's auc: 1\tvalid_1's auc: 0.998081\n",
      "[239]\ttraining's auc: 1\tvalid_1's auc: 0.998098\n",
      "[240]\ttraining's auc: 1\tvalid_1's auc: 0.998081\n",
      "[241]\ttraining's auc: 1\tvalid_1's auc: 0.998087\n",
      "[242]\ttraining's auc: 1\tvalid_1's auc: 0.998081\n",
      "[243]\ttraining's auc: 1\tvalid_1's auc: 0.998081\n",
      "[244]\ttraining's auc: 1\tvalid_1's auc: 0.998087\n",
      "[245]\ttraining's auc: 1\tvalid_1's auc: 0.998098\n",
      "[246]\ttraining's auc: 1\tvalid_1's auc: 0.998098\n",
      "[247]\ttraining's auc: 1\tvalid_1's auc: 0.998098\n",
      "[248]\ttraining's auc: 1\tvalid_1's auc: 0.998103\n",
      "[249]\ttraining's auc: 1\tvalid_1's auc: 0.998125\n",
      "[250]\ttraining's auc: 1\tvalid_1's auc: 0.998119\n",
      "[251]\ttraining's auc: 1\tvalid_1's auc: 0.99813\n",
      "[252]\ttraining's auc: 1\tvalid_1's auc: 0.998125\n",
      "[253]\ttraining's auc: 1\tvalid_1's auc: 0.99813\n",
      "[254]\ttraining's auc: 1\tvalid_1's auc: 0.99813\n",
      "[255]\ttraining's auc: 1\tvalid_1's auc: 0.99813\n",
      "[256]\ttraining's auc: 1\tvalid_1's auc: 0.998125\n",
      "[257]\ttraining's auc: 1\tvalid_1's auc: 0.99813\n",
      "[258]\ttraining's auc: 1\tvalid_1's auc: 0.998119\n",
      "[259]\ttraining's auc: 1\tvalid_1's auc: 0.99813\n",
      "[260]\ttraining's auc: 1\tvalid_1's auc: 0.998136\n",
      "[261]\ttraining's auc: 1\tvalid_1's auc: 0.998136\n",
      "[262]\ttraining's auc: 1\tvalid_1's auc: 0.998136\n",
      "[263]\ttraining's auc: 1\tvalid_1's auc: 0.998141\n",
      "[264]\ttraining's auc: 1\tvalid_1's auc: 0.998158\n",
      "[265]\ttraining's auc: 1\tvalid_1's auc: 0.998136\n",
      "[266]\ttraining's auc: 1\tvalid_1's auc: 0.998119\n",
      "[267]\ttraining's auc: 1\tvalid_1's auc: 0.998136\n",
      "[268]\ttraining's auc: 1\tvalid_1's auc: 0.998147\n",
      "[269]\ttraining's auc: 1\tvalid_1's auc: 0.998136\n",
      "[270]\ttraining's auc: 1\tvalid_1's auc: 0.998141\n",
      "[271]\ttraining's auc: 1\tvalid_1's auc: 0.998152\n",
      "[272]\ttraining's auc: 1\tvalid_1's auc: 0.998163\n",
      "[273]\ttraining's auc: 1\tvalid_1's auc: 0.998163\n",
      "[274]\ttraining's auc: 1\tvalid_1's auc: 0.99818\n",
      "[275]\ttraining's auc: 1\tvalid_1's auc: 0.998174\n",
      "[276]\ttraining's auc: 1\tvalid_1's auc: 0.998174\n",
      "[277]\ttraining's auc: 1\tvalid_1's auc: 0.99818\n",
      "[278]\ttraining's auc: 1\tvalid_1's auc: 0.998196\n",
      "[279]\ttraining's auc: 1\tvalid_1's auc: 0.998191\n",
      "[280]\ttraining's auc: 1\tvalid_1's auc: 0.99818\n",
      "[281]\ttraining's auc: 1\tvalid_1's auc: 0.998201\n",
      "[282]\ttraining's auc: 1\tvalid_1's auc: 0.998196\n",
      "[283]\ttraining's auc: 1\tvalid_1's auc: 0.998196\n",
      "[284]\ttraining's auc: 1\tvalid_1's auc: 0.998191\n",
      "[285]\ttraining's auc: 1\tvalid_1's auc: 0.998223\n",
      "[286]\ttraining's auc: 1\tvalid_1's auc: 0.998212\n",
      "[287]\ttraining's auc: 1\tvalid_1's auc: 0.998196\n",
      "[288]\ttraining's auc: 1\tvalid_1's auc: 0.998185\n",
      "[289]\ttraining's auc: 1\tvalid_1's auc: 0.998174\n",
      "[290]\ttraining's auc: 1\tvalid_1's auc: 0.998174\n",
      "[291]\ttraining's auc: 1\tvalid_1's auc: 0.998163\n",
      "[292]\ttraining's auc: 1\tvalid_1's auc: 0.99818\n",
      "[293]\ttraining's auc: 1\tvalid_1's auc: 0.998185\n",
      "[294]\ttraining's auc: 1\tvalid_1's auc: 0.998191\n",
      "[295]\ttraining's auc: 1\tvalid_1's auc: 0.998191\n",
      "[296]\ttraining's auc: 1\tvalid_1's auc: 0.998196\n",
      "[297]\ttraining's auc: 1\tvalid_1's auc: 0.998218\n",
      "[298]\ttraining's auc: 1\tvalid_1's auc: 0.998212\n",
      "[299]\ttraining's auc: 1\tvalid_1's auc: 0.998212\n",
      "[300]\ttraining's auc: 1\tvalid_1's auc: 0.998218\n",
      "[301]\ttraining's auc: 1\tvalid_1's auc: 0.998212\n",
      "[302]\ttraining's auc: 1\tvalid_1's auc: 0.998191\n",
      "[303]\ttraining's auc: 1\tvalid_1's auc: 0.998207\n",
      "[304]\ttraining's auc: 1\tvalid_1's auc: 0.998196\n",
      "[305]\ttraining's auc: 1\tvalid_1's auc: 0.998196\n",
      "[306]\ttraining's auc: 1\tvalid_1's auc: 0.998196\n",
      "[307]\ttraining's auc: 1\tvalid_1's auc: 0.998185\n",
      "[308]\ttraining's auc: 1\tvalid_1's auc: 0.998191\n",
      "[309]\ttraining's auc: 1\tvalid_1's auc: 0.99818\n",
      "[310]\ttraining's auc: 1\tvalid_1's auc: 0.998196\n",
      "[311]\ttraining's auc: 1\tvalid_1's auc: 0.998201\n",
      "[312]\ttraining's auc: 1\tvalid_1's auc: 0.998191\n",
      "[313]\ttraining's auc: 1\tvalid_1's auc: 0.998185\n",
      "[314]\ttraining's auc: 1\tvalid_1's auc: 0.998207\n",
      "[315]\ttraining's auc: 1\tvalid_1's auc: 0.998196\n",
      "[316]\ttraining's auc: 1\tvalid_1's auc: 0.998207\n",
      "[317]\ttraining's auc: 1\tvalid_1's auc: 0.998201\n",
      "[318]\ttraining's auc: 1\tvalid_1's auc: 0.998201\n",
      "[319]\ttraining's auc: 1\tvalid_1's auc: 0.998201\n",
      "[320]\ttraining's auc: 1\tvalid_1's auc: 0.998196\n",
      "[321]\ttraining's auc: 1\tvalid_1's auc: 0.998185\n",
      "[322]\ttraining's auc: 1\tvalid_1's auc: 0.998191\n",
      "[323]\ttraining's auc: 1\tvalid_1's auc: 0.998185\n",
      "[324]\ttraining's auc: 1\tvalid_1's auc: 0.998185\n",
      "[325]\ttraining's auc: 1\tvalid_1's auc: 0.998174\n",
      "[326]\ttraining's auc: 1\tvalid_1's auc: 0.998174\n",
      "[327]\ttraining's auc: 1\tvalid_1's auc: 0.998169\n",
      "[328]\ttraining's auc: 1\tvalid_1's auc: 0.998185\n",
      "[329]\ttraining's auc: 1\tvalid_1's auc: 0.99818\n",
      "[330]\ttraining's auc: 1\tvalid_1's auc: 0.99818\n",
      "[331]\ttraining's auc: 1\tvalid_1's auc: 0.99818\n",
      "[332]\ttraining's auc: 1\tvalid_1's auc: 0.99818\n",
      "[333]\ttraining's auc: 1\tvalid_1's auc: 0.998185\n",
      "[334]\ttraining's auc: 1\tvalid_1's auc: 0.998212\n",
      "[335]\ttraining's auc: 1\tvalid_1's auc: 0.998196\n",
      "[336]\ttraining's auc: 1\tvalid_1's auc: 0.998201\n",
      "[337]\ttraining's auc: 1\tvalid_1's auc: 0.998207\n",
      "[338]\ttraining's auc: 1\tvalid_1's auc: 0.998207\n",
      "[339]\ttraining's auc: 1\tvalid_1's auc: 0.998218\n",
      "[340]\ttraining's auc: 1\tvalid_1's auc: 0.998229\n",
      "[341]\ttraining's auc: 1\tvalid_1's auc: 0.998229\n",
      "[342]\ttraining's auc: 1\tvalid_1's auc: 0.998223\n",
      "[343]\ttraining's auc: 1\tvalid_1's auc: 0.998218\n",
      "[344]\ttraining's auc: 1\tvalid_1's auc: 0.998207\n",
      "[345]\ttraining's auc: 1\tvalid_1's auc: 0.998191\n",
      "[346]\ttraining's auc: 1\tvalid_1's auc: 0.998196\n",
      "[347]\ttraining's auc: 1\tvalid_1's auc: 0.998207\n",
      "[348]\ttraining's auc: 1\tvalid_1's auc: 0.998196\n",
      "[349]\ttraining's auc: 1\tvalid_1's auc: 0.998201\n",
      "[350]\ttraining's auc: 1\tvalid_1's auc: 0.998207\n",
      "[351]\ttraining's auc: 1\tvalid_1's auc: 0.998218\n",
      "[352]\ttraining's auc: 1\tvalid_1's auc: 0.998229\n",
      "[353]\ttraining's auc: 1\tvalid_1's auc: 0.998229\n",
      "[354]\ttraining's auc: 1\tvalid_1's auc: 0.998223\n",
      "[355]\ttraining's auc: 1\tvalid_1's auc: 0.998223\n",
      "[356]\ttraining's auc: 1\tvalid_1's auc: 0.998229\n",
      "[357]\ttraining's auc: 1\tvalid_1's auc: 0.998229\n",
      "[358]\ttraining's auc: 1\tvalid_1's auc: 0.998229\n",
      "[359]\ttraining's auc: 1\tvalid_1's auc: 0.998234\n",
      "[360]\ttraining's auc: 1\tvalid_1's auc: 0.998218\n",
      "[361]\ttraining's auc: 1\tvalid_1's auc: 0.998218\n",
      "[362]\ttraining's auc: 1\tvalid_1's auc: 0.998218\n",
      "[363]\ttraining's auc: 1\tvalid_1's auc: 0.99824\n",
      "[364]\ttraining's auc: 1\tvalid_1's auc: 0.99824\n",
      "[365]\ttraining's auc: 1\tvalid_1's auc: 0.99824\n",
      "[366]\ttraining's auc: 1\tvalid_1's auc: 0.998245\n",
      "[367]\ttraining's auc: 1\tvalid_1's auc: 0.998256\n",
      "[368]\ttraining's auc: 1\tvalid_1's auc: 0.99824\n",
      "[369]\ttraining's auc: 1\tvalid_1's auc: 0.99824\n",
      "[370]\ttraining's auc: 1\tvalid_1's auc: 0.998251\n",
      "[371]\ttraining's auc: 1\tvalid_1's auc: 0.998256\n",
      "[372]\ttraining's auc: 1\tvalid_1's auc: 0.998278\n",
      "[373]\ttraining's auc: 1\tvalid_1's auc: 0.998283\n",
      "[374]\ttraining's auc: 1\tvalid_1's auc: 0.998273\n",
      "[375]\ttraining's auc: 1\tvalid_1's auc: 0.998262\n",
      "[376]\ttraining's auc: 1\tvalid_1's auc: 0.998256\n",
      "[377]\ttraining's auc: 1\tvalid_1's auc: 0.998262\n",
      "[378]\ttraining's auc: 1\tvalid_1's auc: 0.998251\n",
      "[379]\ttraining's auc: 1\tvalid_1's auc: 0.998245\n",
      "[380]\ttraining's auc: 1\tvalid_1's auc: 0.998267\n",
      "[381]\ttraining's auc: 1\tvalid_1's auc: 0.998278\n",
      "[382]\ttraining's auc: 1\tvalid_1's auc: 0.998289\n",
      "[383]\ttraining's auc: 1\tvalid_1's auc: 0.998273\n",
      "[384]\ttraining's auc: 1\tvalid_1's auc: 0.998273\n",
      "[385]\ttraining's auc: 1\tvalid_1's auc: 0.9983\n",
      "[386]\ttraining's auc: 1\tvalid_1's auc: 0.998316\n",
      "[387]\ttraining's auc: 1\tvalid_1's auc: 0.998322\n",
      "[388]\ttraining's auc: 1\tvalid_1's auc: 0.998322\n",
      "[389]\ttraining's auc: 1\tvalid_1's auc: 0.998311\n",
      "[390]\ttraining's auc: 1\tvalid_1's auc: 0.998311\n",
      "[391]\ttraining's auc: 1\tvalid_1's auc: 0.998316\n",
      "[392]\ttraining's auc: 1\tvalid_1's auc: 0.998322\n",
      "[393]\ttraining's auc: 1\tvalid_1's auc: 0.998322\n",
      "[394]\ttraining's auc: 1\tvalid_1's auc: 0.998316\n",
      "[395]\ttraining's auc: 1\tvalid_1's auc: 0.998311\n",
      "[396]\ttraining's auc: 1\tvalid_1's auc: 0.998327\n",
      "[397]\ttraining's auc: 1\tvalid_1's auc: 0.998322\n",
      "[398]\ttraining's auc: 1\tvalid_1's auc: 0.998322\n",
      "[399]\ttraining's auc: 1\tvalid_1's auc: 0.998333\n",
      "[400]\ttraining's auc: 1\tvalid_1's auc: 0.998333\n",
      "[401]\ttraining's auc: 1\tvalid_1's auc: 0.998344\n",
      "[402]\ttraining's auc: 1\tvalid_1's auc: 0.998338\n",
      "[403]\ttraining's auc: 1\tvalid_1's auc: 0.998333\n",
      "[404]\ttraining's auc: 1\tvalid_1's auc: 0.998338\n",
      "[405]\ttraining's auc: 1\tvalid_1's auc: 0.998333\n",
      "[406]\ttraining's auc: 1\tvalid_1's auc: 0.998333\n",
      "[407]\ttraining's auc: 1\tvalid_1's auc: 0.998316\n",
      "[408]\ttraining's auc: 1\tvalid_1's auc: 0.998322\n",
      "[409]\ttraining's auc: 1\tvalid_1's auc: 0.998327\n",
      "[410]\ttraining's auc: 1\tvalid_1's auc: 0.998327\n",
      "[411]\ttraining's auc: 1\tvalid_1's auc: 0.998333\n",
      "[412]\ttraining's auc: 1\tvalid_1's auc: 0.998327\n",
      "[413]\ttraining's auc: 1\tvalid_1's auc: 0.998333\n",
      "[414]\ttraining's auc: 1\tvalid_1's auc: 0.998344\n",
      "[415]\ttraining's auc: 1\tvalid_1's auc: 0.998349\n",
      "[416]\ttraining's auc: 1\tvalid_1's auc: 0.998344\n",
      "[417]\ttraining's auc: 1\tvalid_1's auc: 0.998355\n",
      "[418]\ttraining's auc: 1\tvalid_1's auc: 0.998349\n",
      "[419]\ttraining's auc: 1\tvalid_1's auc: 0.998355\n",
      "[420]\ttraining's auc: 1\tvalid_1's auc: 0.99836\n",
      "[421]\ttraining's auc: 1\tvalid_1's auc: 0.998365\n",
      "[422]\ttraining's auc: 1\tvalid_1's auc: 0.998344\n",
      "[423]\ttraining's auc: 1\tvalid_1's auc: 0.998344\n",
      "[424]\ttraining's auc: 1\tvalid_1's auc: 0.998355\n",
      "[425]\ttraining's auc: 1\tvalid_1's auc: 0.998349\n",
      "[426]\ttraining's auc: 1\tvalid_1's auc: 0.998365\n",
      "[427]\ttraining's auc: 1\tvalid_1's auc: 0.99836\n",
      "[428]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[429]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[430]\ttraining's auc: 1\tvalid_1's auc: 0.998376\n",
      "[431]\ttraining's auc: 1\tvalid_1's auc: 0.998376\n",
      "[432]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[433]\ttraining's auc: 1\tvalid_1's auc: 0.998365\n",
      "[434]\ttraining's auc: 1\tvalid_1's auc: 0.998355\n",
      "[435]\ttraining's auc: 1\tvalid_1's auc: 0.998344\n",
      "[436]\ttraining's auc: 1\tvalid_1's auc: 0.998349\n",
      "[437]\ttraining's auc: 1\tvalid_1's auc: 0.998382\n",
      "[438]\ttraining's auc: 1\tvalid_1's auc: 0.998382\n",
      "[439]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[440]\ttraining's auc: 1\tvalid_1's auc: 0.998382\n",
      "[441]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[442]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[443]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[444]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[445]\ttraining's auc: 1\tvalid_1's auc: 0.998376\n",
      "[446]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[447]\ttraining's auc: 1\tvalid_1's auc: 0.998376\n",
      "[448]\ttraining's auc: 1\tvalid_1's auc: 0.998382\n",
      "[449]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[450]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[451]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[452]\ttraining's auc: 1\tvalid_1's auc: 0.998365\n",
      "[453]\ttraining's auc: 1\tvalid_1's auc: 0.99836\n",
      "[454]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[455]\ttraining's auc: 1\tvalid_1's auc: 0.99836\n",
      "[456]\ttraining's auc: 1\tvalid_1's auc: 0.998365\n",
      "[457]\ttraining's auc: 1\tvalid_1's auc: 0.998376\n",
      "[458]\ttraining's auc: 1\tvalid_1's auc: 0.998376\n",
      "[459]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[460]\ttraining's auc: 1\tvalid_1's auc: 0.998376\n",
      "[461]\ttraining's auc: 1\tvalid_1's auc: 0.998382\n",
      "[462]\ttraining's auc: 1\tvalid_1's auc: 0.998376\n",
      "[463]\ttraining's auc: 1\tvalid_1's auc: 0.998365\n",
      "[464]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[465]\ttraining's auc: 1\tvalid_1's auc: 0.99836\n",
      "[466]\ttraining's auc: 1\tvalid_1's auc: 0.998365\n",
      "[467]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[468]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[469]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[470]\ttraining's auc: 1\tvalid_1's auc: 0.998376\n",
      "[471]\ttraining's auc: 1\tvalid_1's auc: 0.998376\n",
      "[472]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[473]\ttraining's auc: 1\tvalid_1's auc: 0.998376\n",
      "[474]\ttraining's auc: 1\tvalid_1's auc: 0.998365\n",
      "[475]\ttraining's auc: 1\tvalid_1's auc: 0.998365\n",
      "[476]\ttraining's auc: 1\tvalid_1's auc: 0.99836\n",
      "[477]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[478]\ttraining's auc: 1\tvalid_1's auc: 0.99836\n",
      "[479]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[480]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[481]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[482]\ttraining's auc: 1\tvalid_1's auc: 0.998365\n",
      "[483]\ttraining's auc: 1\tvalid_1's auc: 0.998376\n",
      "[484]\ttraining's auc: 1\tvalid_1's auc: 0.998382\n",
      "[485]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[486]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[487]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[488]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[489]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[490]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[491]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[492]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[493]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[494]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[495]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[496]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[497]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[498]\ttraining's auc: 1\tvalid_1's auc: 0.998382\n",
      "[499]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[500]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[501]\ttraining's auc: 1\tvalid_1's auc: 0.998409\n",
      "[502]\ttraining's auc: 1\tvalid_1's auc: 0.998409\n",
      "[503]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[504]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[505]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[506]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[507]\ttraining's auc: 1\tvalid_1's auc: 0.998409\n",
      "[508]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[509]\ttraining's auc: 1\tvalid_1's auc: 0.998409\n",
      "[510]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[511]\ttraining's auc: 1\tvalid_1's auc: 0.99842\n",
      "[512]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[513]\ttraining's auc: 1\tvalid_1's auc: 0.998426\n",
      "[514]\ttraining's auc: 1\tvalid_1's auc: 0.998431\n",
      "[515]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[516]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[517]\ttraining's auc: 1\tvalid_1's auc: 0.99842\n",
      "[518]\ttraining's auc: 1\tvalid_1's auc: 0.998409\n",
      "[519]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[520]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[521]\ttraining's auc: 1\tvalid_1's auc: 0.99842\n",
      "[522]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[523]\ttraining's auc: 1\tvalid_1's auc: 0.998431\n",
      "[524]\ttraining's auc: 1\tvalid_1's auc: 0.998431\n",
      "[525]\ttraining's auc: 1\tvalid_1's auc: 0.998437\n",
      "[526]\ttraining's auc: 1\tvalid_1's auc: 0.998426\n",
      "[527]\ttraining's auc: 1\tvalid_1's auc: 0.998426\n",
      "[528]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[529]\ttraining's auc: 1\tvalid_1's auc: 0.998437\n",
      "[530]\ttraining's auc: 1\tvalid_1's auc: 0.998437\n",
      "[531]\ttraining's auc: 1\tvalid_1's auc: 0.998442\n",
      "[532]\ttraining's auc: 1\tvalid_1's auc: 0.998442\n",
      "[533]\ttraining's auc: 1\tvalid_1's auc: 0.99842\n",
      "[534]\ttraining's auc: 1\tvalid_1's auc: 0.99842\n",
      "[535]\ttraining's auc: 1\tvalid_1's auc: 0.998426\n",
      "[536]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[537]\ttraining's auc: 1\tvalid_1's auc: 0.998431\n",
      "[538]\ttraining's auc: 1\tvalid_1's auc: 0.998409\n",
      "[539]\ttraining's auc: 1\tvalid_1's auc: 0.998409\n",
      "[540]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[541]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[542]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[543]\ttraining's auc: 1\tvalid_1's auc: 0.998409\n",
      "[544]\ttraining's auc: 1\tvalid_1's auc: 0.998431\n",
      "[545]\ttraining's auc: 1\tvalid_1's auc: 0.998431\n",
      "[546]\ttraining's auc: 1\tvalid_1's auc: 0.99842\n",
      "[547]\ttraining's auc: 1\tvalid_1's auc: 0.998426\n",
      "[548]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[549]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[550]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[551]\ttraining's auc: 1\tvalid_1's auc: 0.998409\n",
      "[552]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[553]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[554]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[555]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[556]\ttraining's auc: 1\tvalid_1's auc: 0.998409\n",
      "[557]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[558]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[559]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[560]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[561]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[562]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[563]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[564]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[565]\ttraining's auc: 1\tvalid_1's auc: 0.998382\n",
      "[566]\ttraining's auc: 1\tvalid_1's auc: 0.998382\n",
      "[567]\ttraining's auc: 1\tvalid_1's auc: 0.998382\n",
      "[568]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[569]\ttraining's auc: 1\tvalid_1's auc: 0.998382\n",
      "[570]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[571]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[572]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[573]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[574]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[575]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[576]\ttraining's auc: 1\tvalid_1's auc: 0.998382\n",
      "[577]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[578]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[579]\ttraining's auc: 1\tvalid_1's auc: 0.998382\n",
      "[580]\ttraining's auc: 1\tvalid_1's auc: 0.998376\n",
      "[581]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[582]\ttraining's auc: 1\tvalid_1's auc: 0.998371\n",
      "[583]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[584]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[585]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[586]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[587]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[588]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[589]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[590]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[591]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[592]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[593]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[594]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[595]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[596]\ttraining's auc: 1\tvalid_1's auc: 0.998382\n",
      "[597]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[598]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[599]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[600]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[601]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[602]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[603]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[604]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[605]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[606]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[607]\ttraining's auc: 1\tvalid_1's auc: 0.998387\n",
      "[608]\ttraining's auc: 1\tvalid_1's auc: 0.998393\n",
      "[609]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[610]\ttraining's auc: 1\tvalid_1's auc: 0.99842\n",
      "[611]\ttraining's auc: 1\tvalid_1's auc: 0.99842\n",
      "[612]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[613]\ttraining's auc: 1\tvalid_1's auc: 0.99842\n",
      "[614]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[615]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[616]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[617]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[618]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[619]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[620]\ttraining's auc: 1\tvalid_1's auc: 0.998398\n",
      "[621]\ttraining's auc: 1\tvalid_1's auc: 0.998404\n",
      "[622]\ttraining's auc: 1\tvalid_1's auc: 0.998431\n",
      "[623]\ttraining's auc: 1\tvalid_1's auc: 0.998426\n",
      "[624]\ttraining's auc: 1\tvalid_1's auc: 0.99842\n",
      "[625]\ttraining's auc: 1\tvalid_1's auc: 0.998415\n",
      "[626]\ttraining's auc: 1\tvalid_1's auc: 0.998442\n",
      "[627]\ttraining's auc: 1\tvalid_1's auc: 0.998437\n",
      "[628]\ttraining's auc: 1\tvalid_1's auc: 0.998442\n",
      "[629]\ttraining's auc: 1\tvalid_1's auc: 0.998453\n",
      "[630]\ttraining's auc: 1\tvalid_1's auc: 0.998442\n",
      "[631]\ttraining's auc: 1\tvalid_1's auc: 0.998447\n",
      "[632]\ttraining's auc: 1\tvalid_1's auc: 0.998447\n",
      "[633]\ttraining's auc: 1\tvalid_1's auc: 0.998442\n",
      "[634]\ttraining's auc: 1\tvalid_1's auc: 0.998431\n",
      "[635]\ttraining's auc: 1\tvalid_1's auc: 0.998431\n",
      "[636]\ttraining's auc: 1\tvalid_1's auc: 0.998431\n",
      "[637]\ttraining's auc: 1\tvalid_1's auc: 0.998447\n",
      "[638]\ttraining's auc: 1\tvalid_1's auc: 0.998447\n",
      "[639]\ttraining's auc: 1\tvalid_1's auc: 0.998458\n",
      "[640]\ttraining's auc: 1\tvalid_1's auc: 0.998453\n",
      "[641]\ttraining's auc: 1\tvalid_1's auc: 0.998447\n",
      "[642]\ttraining's auc: 1\tvalid_1's auc: 0.998447\n",
      "[643]\ttraining's auc: 1\tvalid_1's auc: 0.998453\n",
      "[644]\ttraining's auc: 1\tvalid_1's auc: 0.998453\n",
      "[645]\ttraining's auc: 1\tvalid_1's auc: 0.998458\n",
      "[646]\ttraining's auc: 1\tvalid_1's auc: 0.998464\n",
      "[647]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[648]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[649]\ttraining's auc: 1\tvalid_1's auc: 0.998475\n",
      "[650]\ttraining's auc: 1\tvalid_1's auc: 0.998475\n",
      "[651]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[652]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[653]\ttraining's auc: 1\tvalid_1's auc: 0.998475\n",
      "[654]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[655]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[656]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[657]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[658]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[659]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[660]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[661]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[662]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[663]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[664]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[665]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[666]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[667]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[668]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[669]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[670]\ttraining's auc: 1\tvalid_1's auc: 0.998475\n",
      "[671]\ttraining's auc: 1\tvalid_1's auc: 0.998475\n",
      "[672]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[673]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[674]\ttraining's auc: 1\tvalid_1's auc: 0.998508\n",
      "[675]\ttraining's auc: 1\tvalid_1's auc: 0.998508\n",
      "[676]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[677]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[678]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[679]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[680]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[681]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[682]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[683]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[684]\ttraining's auc: 1\tvalid_1's auc: 0.998475\n",
      "[685]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[686]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[687]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[688]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[689]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[690]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[691]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[692]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[693]\ttraining's auc: 1\tvalid_1's auc: 0.998475\n",
      "[694]\ttraining's auc: 1\tvalid_1's auc: 0.998464\n",
      "[695]\ttraining's auc: 1\tvalid_1's auc: 0.998475\n",
      "[696]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[697]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[698]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[699]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[700]\ttraining's auc: 1\tvalid_1's auc: 0.998508\n",
      "[701]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[702]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[703]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[704]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[705]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[706]\ttraining's auc: 1\tvalid_1's auc: 0.998508\n",
      "[707]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[708]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[709]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[710]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[711]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[712]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[713]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[714]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[715]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[716]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[717]\ttraining's auc: 1\tvalid_1's auc: 0.99854\n",
      "[718]\ttraining's auc: 1\tvalid_1's auc: 0.998535\n",
      "[719]\ttraining's auc: 1\tvalid_1's auc: 0.99854\n",
      "[720]\ttraining's auc: 1\tvalid_1's auc: 0.99854\n",
      "[721]\ttraining's auc: 1\tvalid_1's auc: 0.998551\n",
      "[722]\ttraining's auc: 1\tvalid_1's auc: 0.99854\n",
      "[723]\ttraining's auc: 1\tvalid_1's auc: 0.99854\n",
      "[724]\ttraining's auc: 1\tvalid_1's auc: 0.998535\n",
      "[725]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[726]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[727]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[728]\ttraining's auc: 1\tvalid_1's auc: 0.998535\n",
      "[729]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[730]\ttraining's auc: 1\tvalid_1's auc: 0.998508\n",
      "[731]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[732]\ttraining's auc: 1\tvalid_1's auc: 0.998508\n",
      "[733]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[734]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[735]\ttraining's auc: 1\tvalid_1's auc: 0.99854\n",
      "[736]\ttraining's auc: 1\tvalid_1's auc: 0.998535\n",
      "[737]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[738]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[739]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[740]\ttraining's auc: 1\tvalid_1's auc: 0.998535\n",
      "[741]\ttraining's auc: 1\tvalid_1's auc: 0.998535\n",
      "[742]\ttraining's auc: 1\tvalid_1's auc: 0.998535\n",
      "[743]\ttraining's auc: 1\tvalid_1's auc: 0.998535\n",
      "[744]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[745]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[746]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[747]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[748]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[749]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[750]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[751]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[752]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[753]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[754]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[755]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[756]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[757]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[758]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[759]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[760]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[761]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[762]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[763]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[764]\ttraining's auc: 1\tvalid_1's auc: 0.998508\n",
      "[765]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[766]\ttraining's auc: 1\tvalid_1's auc: 0.998508\n",
      "[767]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[768]\ttraining's auc: 1\tvalid_1's auc: 0.998508\n",
      "[769]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[770]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[771]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[772]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[773]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[774]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[775]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[776]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[777]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[778]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[779]\ttraining's auc: 1\tvalid_1's auc: 0.998508\n",
      "[780]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[781]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[782]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[783]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[784]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[785]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[786]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[787]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[788]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[789]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[790]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[791]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[792]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[793]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[794]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[795]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[796]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[797]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[798]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[799]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[800]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[801]\ttraining's auc: 1\tvalid_1's auc: 0.998508\n",
      "[802]\ttraining's auc: 1\tvalid_1's auc: 0.998508\n",
      "[803]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[804]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[805]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[806]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[807]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[808]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[809]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[810]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[811]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[812]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[813]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[814]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[815]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[816]\ttraining's auc: 1\tvalid_1's auc: 0.998535\n",
      "[817]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[818]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[819]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[820]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[821]\ttraining's auc: 1\tvalid_1's auc: 0.998524\n",
      "[822]\ttraining's auc: 1\tvalid_1's auc: 0.998535\n",
      "[823]\ttraining's auc: 1\tvalid_1's auc: 0.998535\n",
      "[824]\ttraining's auc: 1\tvalid_1's auc: 0.998535\n",
      "[825]\ttraining's auc: 1\tvalid_1's auc: 0.998535\n",
      "[826]\ttraining's auc: 1\tvalid_1's auc: 0.99854\n",
      "[827]\ttraining's auc: 1\tvalid_1's auc: 0.998529\n",
      "[828]\ttraining's auc: 1\tvalid_1's auc: 0.998535\n",
      "[829]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[830]\ttraining's auc: 1\tvalid_1's auc: 0.998519\n",
      "[831]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[832]\ttraining's auc: 1\tvalid_1's auc: 0.998513\n",
      "[833]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[834]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[835]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[836]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[837]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[838]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[839]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[840]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[841]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[842]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[843]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[844]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[845]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[846]\ttraining's auc: 1\tvalid_1's auc: 0.998508\n",
      "[847]\ttraining's auc: 1\tvalid_1's auc: 0.998508\n",
      "[848]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[849]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[850]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[851]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[852]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[853]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[854]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[855]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[856]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[857]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[858]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[859]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[860]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[861]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[862]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[863]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[864]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[865]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[866]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[867]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[868]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[869]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[870]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[871]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[872]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[873]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[874]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[875]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[876]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[877]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[878]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[879]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[880]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[881]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[882]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[883]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[884]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[885]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[886]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[887]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[888]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[889]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[890]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[891]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[892]\ttraining's auc: 1\tvalid_1's auc: 0.998502\n",
      "[893]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[894]\ttraining's auc: 1\tvalid_1's auc: 0.998497\n",
      "[895]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[896]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[897]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[898]\ttraining's auc: 1\tvalid_1's auc: 0.998475\n",
      "[899]\ttraining's auc: 1\tvalid_1's auc: 0.998469\n",
      "[900]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[901]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[902]\ttraining's auc: 1\tvalid_1's auc: 0.998475\n",
      "[903]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[904]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[905]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[906]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[907]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[908]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[909]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[910]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[911]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[912]\ttraining's auc: 1\tvalid_1's auc: 0.998475\n",
      "[913]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[914]\ttraining's auc: 1\tvalid_1's auc: 0.998486\n",
      "[915]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[916]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[917]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[918]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[919]\ttraining's auc: 1\tvalid_1's auc: 0.998491\n",
      "[920]\ttraining's auc: 1\tvalid_1's auc: 0.99848\n",
      "[921]\ttraining's auc: 1\tvalid_1's auc: 0.998475\n",
      "Early stopping, best iteration is:\n",
      "[721]\ttraining's auc: 1\tvalid_1's auc: 0.998551\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "LGBMClassifier(colsample_bytree=0.7, learning_rate=0.02, metric='auc',\n",
       "               n_estimators=1500, num_leaves=30, objective='binary',\n",
       "               random_state=2017, reg_alpha=0, subsample=0.95,\n",
       "               subsample_freq=1)"
      ]
     },
     "execution_count": 151,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x_train,y_train,eval_metric='acc',eval_set=[(x_train, y_train),(x_test, y_test)],early_stopping_rounds=200)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LightGBM预测准确率： 0.9813084112149533\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "y_pred = model.predict(x_test)\n",
    "# y_pred = [1 if i >=0.5 else 0 for i in y_pred]\n",
    "print('LightGBM预测准确率：',accuracy_score(y_test,y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Catboost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {},
   "outputs": [],
   "source": [
    "import catboost as cbt\n",
    "model = cbt.CatBoostClassifier(iterations=5000,learning_rate=0.1,max_depth=7,\n",
    "                               l2_leaf_reg=1,verbose=100,early_stopping_rounds=500,eval_metric='Accuracy')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:\tlearn: 0.9666955\ttest: 0.9567757\tbest: 0.9567757 (0)\ttotal: 8.74ms\tremaining: 43.7s\n",
      "100:\tlearn: 1.0000000\ttest: 0.9789720\tbest: 0.9813084 (24)\ttotal: 568ms\tremaining: 27.6s\n",
      "200:\tlearn: 1.0000000\ttest: 0.9813084\tbest: 0.9813084 (24)\ttotal: 1.1s\tremaining: 26.3s\n",
      "300:\tlearn: 1.0000000\ttest: 0.9813084\tbest: 0.9813084 (24)\ttotal: 1.62s\tremaining: 25.4s\n",
      "400:\tlearn: 1.0000000\ttest: 0.9813084\tbest: 0.9813084 (24)\ttotal: 2.15s\tremaining: 24.7s\n",
      "500:\tlearn: 1.0000000\ttest: 0.9813084\tbest: 0.9813084 (24)\ttotal: 2.66s\tremaining: 23.9s\n",
      "Stopped by overfitting detector  (500 iterations wait)\n",
      "\n",
      "bestTest = 0.9813084112\n",
      "bestIteration = 24\n",
      "\n",
      "Shrink model to first 25 iterations.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<catboost.core.CatBoostClassifier at 0x1e4b8c8b070>"
      ]
     },
     "execution_count": 154,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x_train,y_train,eval_set=(x_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Catboost预测准确率： 0.9813084112149533\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "y_pred = model.predict(x_test)\n",
    "# y_pred = [1 if i >=0.5 else 0 for i in y_pred]\n",
    "print('Catboost预测准确率：',accuracy_score(y_test,y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ligtgbm和catboost不同的模型不同的参数，预测准确率竟然一模一样。。这是个可以研究的问题。。\n",
    "\n",
    "另外树模型不需要规范化"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8.0 64-bit ('Bi_env': venv)",
   "language": "python",
   "name": "python38064bitbienvvenvba07af95a1bb4b078aa8134bba84dff2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.0"
  },
  "toc-autonumbering": false
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
